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ABSTRACT 

This study investigated the educational implications 
of having a group of 17 first-year New Zealand nursing students 
develop a medical expert system. Goals of the study were to determine 
whether, in the process of developing the expert system, the learners 
could: (1) acquire a more in-depth understanding of the specific 
subject domain under study (namely, how to identify alcoholic 
patients); (2) develop a more complex set of problem solving 
strategies; and (3) acquire conditional reasoning skills. An 
etnnographic case study approach was adopted. The learners 
participated as a whole class in six sessions ever a period of 10 
weeks, for a total of 10 contact hours. In addition to whole class 
sessions, they met in small groups to interview subject matter 
experts. Participants then met as a whole class to share their 
i. formation and to construct rules for the knowledge base of the 
expert system. Both quantitative and qualitative measures were used 
to describe how the learners could use computers to acquire knowledge 
and thinking skills in a group learning context where computers were 
used as tools. Data were collected using the following instruments: 
pre- and posttests to measure gains in reasoning abilities; student 
worksheets to solicit information on the participants' problem 
solving strategies and knowledge representation; students' logs; and 
questionnaires to collect information on students' background, 
knowledge acquisition, and attitudes towards the expert system as an 
instructional aid. Due to the preliminary nature of the study and the 
short time frame, no claim was made as to the extent of the 
improvement in problem solving and reasoning skills; however, the 
results clearly indicated that expert systems can be used to enhance 
the development of certain reasoning skills and aiso to facilitate 
the acquisition of a deeper understanding of the subject domain under 
study. (6 references) (GL) 
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Acquiring Expertise and Cognitive Skills in the Process 
of Constructing an Expert System: A Preliminary Study 

Introduction 

The recent advent of microcomputer based expert system shells to 
the classrooms has tremendous educational potential. As suggested by 
some educators (e.g. Romiszowski, 1987), the use of expert systems in 
the tutee mode puts the learners in a problem solving and decision 
making position which results in a deeper understanding of the subject 
domain under study and also in the development of higher order 
thinking skills. However, little research has been conducted to 
document the implementation of expert systems in classrroms; and its 
cognitive and metacognitive implications have not been adequately 
investigated. The present study aimed to investigate the educational 
implications of constructing a medical expert system by a group of New 
Zealand nursing students. More specifically, this study aimed to 
evaluate, in V-^^ t»rocess of developing this expert system, whether the 
learners couid: (a) acquire a more in-depth understanding of the 
specific subject domain under study (namely, how to identify alcoholic 
patients) ; (b) develop a more complex set of problem solving 
strategies; and (c) acquire conditional reasoning skills. It was not 
expected that the finished product would have any commercial values . 
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It is now well documented in the literature that learning takes 
place when a learner engages actively in the construction of his/her 
knowledge (Shuell, 1986) . An important task for the teachers is 
therefore to provide a learning environment which facilitates the 
active acquisition of knowledge. Classroom expert system oriented 
projects could be conducive to the active acquisition of knowledge and 
skills by the learners . To build an expert system, learners need to 
understand how knowledge is structured in the computer program. As 
well/ in order to solicit expert information to construct the 
knowledge base, learners need to elaborate on the problem, evaluate 
the goals and sub-goals to be achieved, and monitor the knowledge 
needed to achieved these goals. Furthermore, to utilize the expert 
information obtained from interviews, they have to interpret and 
understand how the experts represent and use their subject domain 
knowledge to solve che problem. This process forces the novice 
learners to reflect upon their own existing shallow knowledge 
structure and create new structure to accommodate the multilevel 
expert knowledge (Bereiter & Scardamalia, 1986) . The rule based 
structure of the expert system shell provides a cognitive tool for the 
learners to structure the new knowledge. This whole exercise 
facilitates a more in-depth understanding of the specific knowledge 
domain under study, as well as the acquisition of the more complex and 
efficient problem solving strategies. Also, since conditional rules 
(If.., then) are used to represent knowledge in the expert system, the 
exercise could enhance the conditional reasoning ability of the 
learners . 
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Method 

Subjects 

17 first year nursing students (15 females^ 2 males) from a New 
Zealand city polytechnic participated voluntarily in this study as 
part of their computer awareness course. Prior to this study they had 
received instruction in word processing and database management, but 
had no training in the use of expert systems. They had no formal 
training in logic or thinking skills. 1 students dropped out during 
various stages of the study and data collected from 10 students were 
included in the final analysis. 

Instruments 

Micro-Prolog and an expert system shell called Augmented Prolog 
for Expert Systems (Apes) running on the IBM PC were used in this 
study. Micro-Prolog, a computer language based on predicate logic, is 
a version of Prolog (PROgramming in LOGic) developed for interactive 
use on microcomputers (Micro-Prolog Reference Manual, 1985) . Apes 
augments micro-Prolog with features which are characteristic of expert 
systems and therefore could be used as an expert system shell for 
constructing logic based expert systems (Hammond & Sergot, 1984) . 
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Procedures 

The learners participated as a whole class in six sessions over a 
period of 10 weeks in 1988, for a total of 10 contact hours. Other 
than the whole class sessions, they also met in small groups to 
interview the experts and also worked on the interview transcripts for 
3 to 4 hours. In the first three sessions instruction was given to 
the participants on the operation of the expert system shell and on 
general problem solving strategies. An expert was also invited to 
talk about alcoholism so that the participants had some background 
knowledge on the subject domain before conducting their interviews. 
Reading materials on interviewing techniques were also provided. 
After having interviewed the experts the participants met as a whole 
class to share their information and construct rules for the knowledge 
base of the expert system. The rules were coded by the researcher in 
order to save time. Then the expert system was implemented and 
validated by the whole class. 

Data Collection 

An ethnographic case study approach was adopted in this study. 
Both quantitative and qualitative measures were used to describe how 
the learners could use computers to acquire knowledge and thinking 
skills in a group learninvj context. Data were collected in the 
following ways: 

1. Reasoning Tests - Two parallel sets of conditional reasoning 
tests were administered to the participants, one at the 
beginning and the other immediately after the study to 
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measure gains in reasoning abilities. Eight forms of 
conditional arguments were included in these tests (see 
Figure 1) . 

Figure 1: Eight Forms of Conditional Arguments 
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In each test there were 64 sets of three statements, most of 
them selected from popular logic textbooks (Taplin & 
Staudemnayer, 1973) . For example, a typical question might 
have been: 

(a) If I am X, then you are Y, 

(b) I am X. 

(c) You are Y. 

The participants were told that statements a and b were known 
to be true and they were required to evaluate whether 
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statement c necessarily followed from statements a and b. 
They were also asked to state how confident they were with 
their answers. 

2. Student Worksheets - Three student worksheets were 
administered to solicit information on the participants' 
problem solving strategies, how they represented their 
knowledge/ and their understanding of the experts' knowledge 
structure . 

3. Students' Logs - Each student was requested to keep a log 
recording what they had learned in each session. 

4. Questionnaires - A questionnaire was administered at the 
beginning of the study to collect background information. 
Another questionnaire was administered at the end of the 
study to collect information on knowledge acquisition and 
also on the learners' attitudes towards the use of the expert 
system as an instructional aid. 

Results and Discussions 

Knowledge Acquisition 
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One of the objectives of this study was to investigate whether 
learners could acquire a deeper understanding of alcoholic problems, 
especially on how to identify alcoholic patients in a hospital ward by 
constructing an expert, system. At the end of the study a simple 
expert system to assist the learners to identify alcoholic patients 
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was built. This expert system had 35 rules and exhibited a 
hierarchical knowledge structure (see Figure 2) . 

Figure 2: Knowledge Base of the Expert System 
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This expert system was a rule-based system where knowledge was 
represented by rules. In micro-Prolog, a rule consists of a 
conclusion and at least one condition. It has the form: 

Conclusion if condition and condition . . . 

Users could query the expert system by asking a question. For 
example: 

Conf inn(Peter is an alcoholic) 

The inference mechanism of the system would activate (fire) the rules 
and perform the deduction (inference) . The conditions of the rules 
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were evaluated and additional information were sought from the users. 
The users were asked for whe medical history of the patient^ and also 
the symptoms and related illnesses the oatient had. Based on these 
responses^ the system could deduce an answer and report on the status 
of the patient. Figure 3 lists a sample of these rules. 

Figure 3: Sample Rules of the Expert System 

X is most likely an alcoholic if 

X has previous history of dependency and 
X has related illnesses and 
X has symptoms 

X is probably an alcoholic if 

X has related illnesses and 
X has symptoms 

X has related illnesses if 

X has liver damage and 
X has memory defects and 
X has infections 

Representing expert knowledge in rules and breaking them down 
into inter-related components required more than a surface 
understanding of the subject domain . From the rules they have 
constructed/ it was evident that the participants of this study had 
acquired the knowledge of identifying alcoholic patients. It was 
further evident from the participants' verbal reports and responses to 
the worksheets and questionnaire administered at various stages of the 
study that they could better comprehend the complexity of the problem 
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and had a deeper understanding of the subject domain by constructing 
the expert system. They could also reflect upon the knowledge 
structure of the experts and elaborated on their own knowledge 
structure. There was also a clear indication of a deeper level of 
understanding on the subject domain when the participants were asked 
to design a similar course to teach their fellow classmates at the end 
of the study. Of those who had completed the study/ specific 
guidelines as to t\e course content and strategies to be employed were 
reported. One of the participants gave the following outline: 

'1. Firstly give several definitions on alcoholism so th^t 

the student finds a full detailed definition they can use. 

2. Identify reasons possible for alcohol abuse to take place. 

3. Then discuss signs and symptoms of alcoholism giving 
them a ranking from most common and likely [to] least 
common . 

4. Discuss permanent damage done by alcohol abuse. 

5. Talk about method of diagnosis used when dealing with 
alcoholism. 

6. Investigate treatments available to help an alcoholic/ 
and long term measures that are/need to be taken to 
prevent alcohol abuse happening again.' 

From this course outline it was clearly shown that this 
participant could systematically break down the course content into 
related sub-topics. Apparently/ her level of understanding of the 
course structure was shared by another participant who noted that in 
designing the course on alcoholism the "main problems [should be] 
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broken down into sub-goals showing inter-relationship between 
factors". She further explained that "breaking down a problem 
facilitates understanding, and linking factors helps objectivity, i.e. 
working out assessment priorities for treatment.. " 

On the other hand, those participants who did not complete the 
study only gave partial and superficial answers to these questions 
which reflected their lack of comprehension of the complexity of the 
problem. They also did not use any specific problem solving 
strategies in planning the lessons. 

Learning did not take place only in the actual construction of 
the expert system. Interviewing the experts and group discussions 
also facilitated learning. For one participant, transcribing the 
interview tapes was a learning experience. 

'During transcription of taped interview I felt I only 
then began really understanding alcoholism - going back 
and back over gives you a i;trong picture on the subject 
and pusnes you further to ask questions.' 

It seemed that the process of interviewing the experts had forced 
the participants to think about the problem carefully so as to be able 
to ask relevant questions and respond to the experts' answers . 
Transcribing the interview tapes also allowed the participants the 
opportunity to think about the problem more deeply. 
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Problem solving strategies 

It was noted in this study that in order to construct rules for 
the expert system, the participants need to break down the expert 
knowledge into components and also need to build a hierarchy of 
inter-related knowledge structure. In this proce^iJS not only could the 
participants acquire a deeper understanding of the subject domain, and 
also elaborate their own knowledge structure, but could also acquire 
some general problem solving skills in organizing and categorizing 
information. It was clear from the worksheets and questionnaire that 
these skills had been enhanced. When the participants were asked what 
procedures/strategies would they follow when they solved problems, the 
following reports were made: 

'Identify the goal... If the goal was too large, break it down 
into subgoals and work up from there.' 

'A hierarchy. . .use of sub-groups, linking, investigating, 
valisiating ' 

'...look closely at the problem. . .making sure fully 
understand the meaning of it... seek further knowledge 
to relate ideas of possible solutions being iiopropriate 
using the supplied information. .. I would break it up into 
the simplest form. ..I would endeavour to relate all the 
ideas... in working towards a goal.' 

'Break it down in a logical manner by dividing it into 
sections and then checking to see if these are 
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inter-relating factors. . 

With regard to the transfer of skills, the participants were 
asked whether what they had learned in the study would be useful in 
other subject domains. They reported that since the study was a 
problem solving exercise it would be useful in other domains. One 
participant noted: 

'...the problem solving strategies we have learned fit well 
into an important part of nursing practical care called 
nursing process ... can be a foundation for our use...' 

And they also realized the importance of thinking (understanding) 
in the learning process. One participant noted: 

'I have learnt how easy it is to take so much information 
for granted - in lectures you can see it/hear it/ it's 
just rote learning ... it' s not really thinking ana it's 
when you're truly actively using your thinking. . .that 
you can remember something because you had to think 
about the topic. When you understand something only 
then can you add on and build onto the previous 
knowledge then it has significance.' 

And the exercise helped them to think. Another participant wrote: 

'...it has given me the freedom to think clearly, to 
really look at something and make myself .. .check 
all the possibilities not just view it literally...! 
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found that y 



can look at problems framing a 



pyramid analogy, you don't have to stick to one 



method for a life time, many ways to do it.' 



Which was shared by a third one: 



'...it allows you to have an open mind approach to 



'^taer problems and to other's problems.' 



Conditional Reasoning 

As reported in the previous section, the expert system designed 
in this study was a rule-based logical system. The rules in the 
expert system were all conditional arguments. Constructing these 
rules and subsequently validating and querying the system was a good 
training exercise in conditional reasoning for the participants. The 
explanations provided by the system made the reasoning process more 
transparent and further enhance understanding. A typical example of 
querying the system might have been: 



user: Peter sees doctor? 

expert system: Is it true that Peter is sick? 

user: Yes 

expert system: Yes, I can confirm that Peter sees doctor 

user: why? 

expert system: To deduce Peter sees doctor 
I used the rule 



rule : 



If X is sick/ then X sees doctor 
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X sees doctor if X is sick 
I can show Peter is sick 

user: Peter sees doctor? 
expert system: Is it true that Peter is sick? 
user: No 

expert system: No, I cannot confirm Peter sees doctor 
u3er: »*hy? 

expert system: In trying to prove Peter sees doctor 
I tried the rule 
X sees doctor If X is sick 
I cannot show Peter is sick 



What the participants could learn from the explanations given by 
the system was that in a conditional argument when the antecedent 
(Peter is sick) was affirmed, it would be valid to affirm • the 
consequence (sees doctor) . But when the antecedent was denied, it 
would not be valid to affirm or deny the consequence (whether Peter 
sees his doctor) . The feedback provided by the expert system could 
stimulate a lot of discussion and the relationship between the 
conditions and conclusions of conditional arguments would be made 
clear to the learners. 

In this study two parallel sets of reasoning tests were designed 
to measure the effects of this exercise on the conditional reasoning 
abilities of the participants . A paired t-test was conducted on each 
of the eight forms of conditional arguments (eight questions each) in 
the two reasoning tests. Significant improvements were identified in 
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forms 4 (t»2.51 and p<0.05) and 7 (t=5.66 and p<0.01). It was noted 
that the form 4 conditional argument (denying the antecedent f 
affirming the consequence) was one of the two forms that the 
participants had difficulty to understand and scored poorly (the other 
one was form 5, affirming the consequence^ affirming the antecedent) . 
Although the improvement in this form of argument was not drastic 
(average pretest score = 0.67, posttest score = 2.8) , judging from the 
limited amount of time the participants had spent in using the oxpert 
system (about 5 hours), the improvement was impressive. The 
participants also showed improvement in form 5 (pretest score » 1.3, 
posttest score = 1.8), but the result was not significant (t=0.43, 
p-0.667) . 

Conclusions 



This study proposed a computing learning environment conducive to 
the acquisition of subject domain knowledge and certain problem 
solving and reasoning skills. An ethnographic case study approach was 
adopted to describe the learning process taken place in a group 
setting where computers were simply used as tools. The focus was not 
in developing a product, in this case an expert system, but the 
process in which learners could interact with one another in deepening 
their understanding of alcoholism. Due to th'^ preliminary nature of 
this study and also the short period of time no claim was made 
regarding the extent of the improvement in problem solving and 
reasoning skills in this study. However, it was clearly seen from the 
results that expert systems could be used to enhance certain reasoning 
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skills, and also in the acquisition of deeper understanding of t^e 
subject domain under study. 

The use of expert systems in the classrooms has tremendous 
promise in the acquisition of expertise and in the teaching and 
learning of problem solving skills. However, it is noted from this 
study that the instructional procedures employed in an expert system 
oriented project need to be carefully designed so that the 
participants could have a sense of control and ownership throughout 
the learning process. It is also very important to integrate the 
expert system project into the participants' curriculum so as to 
facilitate applications. To allow adequate preparation for the 
interviews and to assimilate and elaborate on the knowledge acquired 
required a lot of discussions which may not be feasible if such a 
project is only treated as an extra-curricular activity. It is labour 
intensive and time consuming, especially when the participants have 
little computing experience and knowledge on the subject domain. The 
major limitation of this preliminary study was the inadequate 
attention paid to the time factor. If the study was conducted when 
the participants' final examinations were farther away, the drop out 
rate would have been greatly reduced. Also, had it not been due to 
the inadequate amount of time spent in various stages of the study, 
e.g, in the actual construction of the expert system, cognitive 
outcomes could be oetter documented and a more sophisticated expert 
system could have been built. 
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